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Abstract. Different from face verification, face identification is much 
more demanding. To reach comparable performance, an identifier needs 
to be roughly N times better than a verifier. To expect a breakthrough 
in face identification, we need a fresh look at the fundamental build¬ 
ing blocks of face recognition. In this paper we focus on the selection 
of a suitable signal representation and better matching strategy for face 
identification. We demonstrate how Gabor phase could be leveraged to 
improve the performance of face identification by using the Block Match¬ 
ing method. Compared to the existing approaches, the proposed method 
features much lower algorithmic complexity: face images are only filtered 
by a single-scale Gabor filter pair and the matching is performed between 
any pairs of face images at hand without involving any training process. 
Benchmark evaluations show that the proposed approach is totally com¬ 
parable to and even better than state-of-the-art algorithms, which are 
typically based on more features extracted from a large set of Gabor 
faces and/or rely on heavy training processes. 


L Introduction 

Face recognition as an active research topic in pattern recognition has been 
intensively studied for more than two decades [niaEiiisiEiMiiig. Face recog¬ 
nition works in two essentially different modes: face verification and face iden¬ 
tification. Face verification performs 1:1 matching to provide a binary decision 
on the claimed identity. The performance of face verification in controlled sce¬ 
narios has reached a rather high accuracy m- Recently, more research efforts 
have been devoted to face verihcation in unconstrained conditions. By utilizing 
strong alignment approaches miaiss], pose-robust matching schemes [HIIHIIIS], 
and advanced deep-learning techniques [T6l[28l[29], significant improvement in 
the performance of face verification has been achieved. The performance on the 
benchmark, the Labeled Face in the Wild (LFW) dataset [TS], is close or even 
go beyond that of human beings [3S] . 

In contrast, face identification is more difficult. It performs 1:N matching 
to sort out the gallery images based on the pair wise similarity measurement. 
Obviously, the operating requirement in face identification is vastly more de¬ 
manding than operating merely in verification: an identifier needs to be roughly 
N times better than a verifier to achieve comparable odds against making false 
matches [3]. This is probably the reason why the progress that has been made 
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in face identification was not significant in the last five years. Even though the 
proposed approaches were more and more complex, the recognition performance 
on the benchmark evaluation is almost the same [3j[5j[34l[35]. To make a break¬ 
through in face identification, it seems that we have to revisit the basis of face 
recognition, and have a fresh look at the fundamental building blocks of face 
recognition. 

In face identification, one of the most basic building blocks is the construction 
of features for measuring the similarity between two face images. The construc¬ 
tion of features consists of two steps: (1) selection of a suitable representation for 
face images; (2) feature extraction from the representation. There is a large col¬ 
lection of research papers on how to extract stable, local or global discriminative 
features, e.g. the commonly used SIFT [50], HOG [5], and LBP [T]. Recently, 
these features are criticized because they are ‘hand-crafted’. It was claimed that 
better features can learned from big data collections through deep-learning ap¬ 
proaches [TOII5R1I50] . But due to inherent complexity of face identification, there 
is no literature on using deep-learning to learn features for face identification. 
On the other hand, a significant progress has been made in the selection of more 
suitable representation of faces for face identification. Compared to using the raw 
image pixels, feature extraction from the amplitude or phase spectrum of the 
Gabor filtered images has shown a significant improvement in the performance 
of face identification. Although the complexity is raised 40 times or even more, 
the identification rate in benchmark evaluations is improved for more than 20%, 
reaching around 90% [13]. This is due to the so-called ‘blessing of dimension¬ 
ality’. Now the question is how to achieve face identification rate from 90% to 
95%, or even higher. In this paper we argue that Gabor phase could enable such 
performance improvement for face identification. 

Most existing face identification algorithms extract discriminative features 
from the Gabor amplitude. The major advantage of the Gabor amplitude is that 
it varies slowly with the spatial position so that it is robust to texture variations 
caused by dynamic expressions and imprecise alignment. But the problem is 
Gabor amplitude depends on the imaging contrast and illumination |I0| . This 
means that if two photos were taken at two times in different environments 
(unfortunately, this is the most common practice of face identification that the 
probe and gallery images were taken in two occasions), it is hard to extract 
consistent features from the Gabor amplitude. 

Alternatively, the Gabor phase can be utilized since it is robust to light 
change. In fact, it has been well-known for a long time that phase is more im¬ 
portant than amplitude from the signal reconstruction point of view (see |22]1. 
Gabor phase should have played more important role in face identification. How¬ 
ever, the use of Gabor phase in face recognition is far from being common and 
successful: it often had worse or nearly the same performance as the amplitude 
in comparative experiments [3l[T2ll34jl40] . This is largely due to two challenging 
issues: (1) Gabor phase is a periodic function and a hard quantization occurs 
for every period; (2) it is very sensitive to spatial shift [ 32111 D], which imposes 
a rigid requirement on face image alignment. The first issue was partly solved 
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by introducing the phase-quadrant demodulation technique m, but the second 
one is still far from being solved. The state-of-the-art Gabor phase approach 
(LGXP [34]) extracts varied LBP from the phase spectrum. Since the combi¬ 
nation of the phase and LBP is also sensitive to the spatial shift, the power of 
Gabor phase does not demonstrate in face identification. 

In this paper, we propose a method that merely leverages the power of Ga¬ 
bor phase to conduct face matching for face identification. By using the Block 
Matching method our approach does not depend on training process or fu¬ 
sion of any other features. Benchmark evaluations on the FERET show that the 
identification rate reaches 95% on the hardest ‘Dup2’, which is the best result 
reported in the literature so far. 


2 Related work 

In this section, we first briefly review the Gabor representation and recent Ga¬ 
bor based methods that utilizing the Gabor amplitude or phase in different 
ways. Then we introduce the Block Matching method, which features a different 
matching strategy from the most existing approaches, used in this work. 

2.1 Gabor Wavelet and Related Approaches 

A Gabor face is obtained by filtering the image with the Gabor filter function, 
which is defined as: 


where u and v define the orientation and scale of the Gabor kernels respectively, 
and the wave vector is defined as: 

r. _ pif^u 

where ky = kmax!P, (t’u = utt/S; kmax is the maximum frequency, a is the rela¬ 
tive width of the Gaussian envelop, and / is the spacing factor between kernels 
in the frequency domain m- The discrete filter bank of 5 different spatial fre¬ 
quencies (u G [0, • • • , 4]) and 8 orientations {u £ [0, • • • , 7]) are mostly exploited 
to filter the face images to facilitate multi-scale analysis. 

To extract the Gabor features, one popular way is to extract the LBP-type 
patterns from the complex Gabor transformed image. As in [33], the LGBP 
feature is extracted from the amplitude spectrum. In |38| and [34] , 4-ray phase- 
quadrant demodulator is applied to demodulate the phase from each complex 
Gabor coefficients, and local binary phase descriptors are subsequently generated 
from the demodulated phase spectrum. Gabor local descriptors can also be built 
by applying dimension reduction technique (e.g. ELD) to the raw Gabor ampli¬ 
tude patches as in m, or by computing the local representation exemplified as 
GOM [^ and SLF [35] . 
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Fusing the features that are independent of the local Gabor features can also 
lead to better performance: dZlEniEH] fuses the global (holistic) features with 
local ones on feature level; [34] proposes fusion of Gabor phase and amplitude 
on the score and feature levels; |5] fuses real, imaginary, amplitude and phase. 
Alternatively, attaching illumination normalization step and weighting the local 
Gabor features is shown to be helpful as well |3]. 


2.2 The Block Matching method 

While most local matching approaches perform matching only between the spa¬ 
tially corresponding patches, some others allow each segmented patch of one 
image to search the best matching from spatially neighboring locations on the 
other image to achieve better robustness to the spatial shift of the textures as 

in 01171113111]. 

The Block Matching method proposed by [Anonymous authors] jH] exploits 
the searching strategy to find best patch-wise matches, and during the same 
process, performs pair-specific normalization to compute the patch-wise distance. 
It can conduct face matching with any two face pair at hand without training. 
Although only raw pixel intensity was used, the evaluations show that the Block 
Matching outperformed the LBP method and its improved high-order variants 

m- 

3 The Gabor Phase Block Matching Approach 

Although feature descriptors and similarity metrics are different, most of the 
existing local based methods perform matching of local features only between 
spatially corresponding patches. This means that the matching philosophy em¬ 
bedded in most of the local based approaches by default sees the spatially cor¬ 
responding patches/features as the best match (since they perform matching 
only between spatially corresponding local regions). It might be true when two 
faces have different shapes and configurations, but due to the movement of fa¬ 
cial components, head pose or imprecise alignment, the spatially corresponding 
patches would cover different face regions (as Fig. 2 in (13 )■ That is, due to the 
sensitiveness to the spatial shift of Gabor phase, the best matching regions are 
likely located away from the corresponding locations even if the matching face 
pairs are aligned. A better solution is to utilize searching scheme that enable 
each region to search for the best matching. In other words, to fairly match the 
local phase descriptors it is intuitive to combine a searching strategy that en¬ 
ables each local phase patch to search for its best match from the neighborhood 
locations. 

Based on this thinking, we adopt the Block Matching method [31] for match¬ 
ing Gabor phase . The matching process of our Gabor Phase Block Matching 
(GPBM) is illustrated in Figjljwith the details of the Block Matching searching 
scheme shown in Fig|3 A two-step matching process is conducted for matching 
a probe-gallery face pair. First, the face images are transformed to Gabor space 
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Step 1, Gabor Filtering and Phase Quantization 


Step 2. Block Matching method 



Fig. 1. Gabor Phase Matching using the Block Matching method 


using two Gabor filters of 1 scale and 2 orientations. The filtered images are de¬ 
modulated by a Gray-coded Phase Shift Keying (PSK) demodulator to quantize 
the phase. Second, the demodulated phase spectra are inputted to the Block 
Matching method m to determine the pair wise distance between a probe (pb) 
image and a gallery (gl) image. 

Specifically in Step 2 of Fig.Hl we first segment the probe phase spectrum into 
N non-overlapping patches and each of the patches is simply formed 

by the raw phase code of the block. For each probe patch centered at image 
coordinate (a:„, yn) (noted as {xmVn)), h searches within the corresponding 
searching window and yields a patch-wise distance vector which is noted as: 




( 1 ) 


where L is the number of candidate gallery patches within the (2i?-|-l) x (217-1-1) 
searching window, i.e. L = (2i?-|-1) • (2(7 -|-1) when applying full search method, 
R and C stands for the searching offset in vertical and horizontal directions 
respectively. Each element in is computed by performing explicit matching 
over the raw demodulated phase as: 


XOR(/(P'’^ {Xn, Vn), decimal 


( 2 ) 


where the patch-wise distance metric is the 2-norm of element wise Humming 
distance in decimal and {xi,yi) denotes the coordinate of the patch center within 
searching window on the gallery face image so that, 


Xi = Xn-\- ^x, Ax G [—(7, (7] 
yi=yn + Ay, AyG[-R,R]. 


We then calculate the slope of the linear fitting of the first 5 ascendingly 
sorted values of Dn for normalization of the patch wise distance for each patch, 
such that the preliminary normalization factor is 




kn/d 


n? 


(4) 
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Fig. 2. The Block Matching scheme 


where dn = min{Dn)- It is further normalized as: 

N-l 

Sn = Sn/ ^ ^ Sn- (5) 

n—0 

Finally, the distance between a matching pair of probe and gallery face is 

N-l 

= y] (s„ • dn). (6) 

n=0 


4 Experiments and Results 

4.1 Database Selection 

There are a variety of large-scale datasets available for benchmark evaluation of 
different face recognition approaches, such as the FERET [21], ERGC 2.0 [25] 
and the LEW dataset. But the ERGC2.0 and LEW are dedicated face veri¬ 
fication benchmarks. Thus we select the EERET database which is the most 
commonly recognized face identification benchmark to evaluate and compare 
our method with state-of-the-art approaches. In addition, the GMU-PIE [25] 
dataset is selected to evaluate our GPBM under variations of pose, expression 
and illumination in the probe images. 


4.2 Experimental Setup 

Face images are first normalized based on the positions of both eyes and the 
Gabor filtered face image to 150 x 136 so that the same height/width ratio (1.1 
: 1) is maintained the same as in [34ll37] . We use a 6 x 6 Gabor filter with v = 0, 
u € {2,6}, / = v^, kmax = 7’'/2, CT = 27r. Accordingly, Gray-coded 16-PSK 
demodulator is used for phase quantization and the constellation is shown in 
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Fig. 3. 16-PSK Demodulator Constellation 
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Fig. 4. Recognition Rates Under Different Parameters 








(a) H15W1SR4C4 (b)H16W16R4C4 (c) H16W12R4C4 

(d) H21W15R5C4 (e) H25W17R6CS (f) H2dW19R7C6 

(g) H30W19R7C6 
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Parameter Settings 


4.3 Evaluations on the CMU-PIE database 

The CMU-PIE database contains 41368 images of 68 subjects. Images with Pose 
Label 05, 07, 09, 27, and 29 under 21 illuminations (Flash 2 to 22) of all the 68 
persons are selected as the probe set. 

In the Blocking Matching method, the most important parameters are the 
block size {H and W) and searching offset {R and C). We have conducted a set of 
empirical tests over other datasets to select suitable parameters. We found that 
H and W satisfying R fv max{H, VF)/4, R/C ~ 1.2 and H/W ~ 1.5 work quite 
good. In our experiments, we select the block size of 29 x 19 and search offset of 
R = 7, C = 6 pixels (a tradeoff between performance and complexity). For this 
group of parameters we selected the first 2000 probe images to evaluate its effect 
on the performance. In addition, we check how sensitive the performance is to 
the selection of parameters by looking around the selected group of parameters. 
The test results are shown in Fig. S) From the test results one can see that the 
performance is quite robust to the selection of parameters. 

We then conduct experiments on the CMU-PIE probe set and compare our 
GPBM with G_LBP and G_LDP [23 ■ The G_LBP is the Gabor version LBP 
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and the G_LDP is a type of improved Gabor amplitude Local Binary Pattern. 
The G_LDP achieved equivalent performance as LGXP (Gabor Phase pattern) 
on the FERET evaluations so that it is a good reference for comparison. The 
comparative rank-1 recognition rates are listed in Table [T] It can be seen that 
our method is at least 3% better than the G_LDP, even though LDP extract 
much more effective patterns than the LBP from the Gabor amplitude space. 
Utilizing the Gabor phase in the Block Matching scheme is more effective in 
dealing with pose and illuirrination changes than LBP-type patterns extracted 
from the Gabor amplitude space. 


Table 1. Comparative rank-1 recognition rates of GPBM on the CMU-PIE database 


Method 

Accuracy 

GJLBP * 

71% 

GJLDP 2nd-order * 

72% 

GJLDP 3rd-order * 

79% 

GJLDP 4th-order * 

74% 

GPBM 

82% 


* The recognition rates are estimated from Fig. 12a in [37] . 


4.4 Evaluations on the FERET database 

The FERET database contains 1196 frontal images in the gallery set, 1195 im¬ 
ages with different expressions in the probe set ‘Fa’, 194 images with illumination 
variations in the probe set ‘Fc’, 722 images taken in later time in the ‘Dupl’ set, 
and 234 images taken at least 1 year later than the gallery set form the hardest 
‘Dup2’ set. We faithfully follow the evaluation protocol of the FERET dataset. 
The results of our GPBM with other approaches using Gabor-phase are listed 
in Tabled 

From Table[2]we can see that in a fair comparison, where only Gabor phase is 
utilized for matching, our GPBM is almost 12% better than LGXP in the hardest 
‘Dup2’; even in unfavorable comparisons, where pre-processing, training, and fu¬ 
sion method are exploited by LMGEW//LN-I-LGXP and S[LGBP Mag-|-LGXP], 
our GPBM still outperforms. To our best knowledge, the S[LGBP_ Mag-I-LGXP] 
— aided by the Gabor amplitude and training procedures — was state-of-the-art 
Gabor phase based method in the hardest FERET ‘Dup2’, and our GPBM is 
better than that. 

We also comprehensively compare our GPBM with other recent state-of-the- 
art approaches based on other techniques on the FERET in Table |3l From the 
table one can see that all these approaches are based on Gabor features, which 
indicates the Gabor space is a very effective signal representation. Our GPBM 
method outperforms all the other approaches on the hardest ‘Dup2’ set and it 
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Table 2. Comparative rank-1 recognition rates of Gabor-phase based approaches on 
the FERET database 


Method 

Fb 

Fc 

Dupl 

Dup2 

LGBP_Pha [10] 

93% 

92% 

65% 

59% 

LGBP_Pha““®'**'’‘* go] 

96% 

94% 

72% 

69% 

J3gj 

97.5% 

99.5% 

79.5% 

77.8% 

LGXP [34] 

98% 

100% 

82% 

83% 

LGXP-fiBFLD gl] 

99% 

100% 

92% 

91% 

S[LGBP_Mag-fiLGXP] [34] 

99% 

100% 

94% 

93% 

LMGEW//LN-fiLGXP [3] 

99.9% 

100% 

94.7% 

91.9% 

GPBM 

99.4% 100% 95.3% 94.9% 


Table 3. Comparative Summary with Recent State-of-the-art Face Identification Ap¬ 
proaches 


Methods 

Image Size 

Photometric 

Processing 

Gabor Filter 
Scale Orient. 

Gabor Feature 
Space 

Training Data 
Independent 

R,ank-1 R,ate 
on FERET Dup2 

LGXP EJ 

88 X 80 

No 

5x8 

Phase 

No 

83% 

S[LGBP+LGXP] EU 

88 X 80 

No 

5x8 

Amplitude + Phase 

No 

93% 

GOM El 

160 X 128 

No 

5x8 

Amplitude + Phase 

No 

93.1% 

LN+LGXP El 

251 X 203 

Yes 

5x8 

Phase 

No 

91.9% 

LN+LGBP El 

251 X 203 

Yes 

5x8 

Amplitude 

No 

93.6% 

SLF-RKRJ2 E3 

150 X 130 

No 

5x8 

Amplitude 

No 

94.4% 

GPBM 

150 X 136 

No 

1 X 2 

Phase 

(explicit matching) 

Yes 

94.9% 


features three advantages: 1) it only requires two Gabor filters, which is 20 times 
less than other methods which utilized 40 Gabor filters; 2) it does not require 
any training procedure or any attached database to perform face matching; 3) 
it uses merely raw phase for matching and no feature extraction was performed. 

The computational complexity is always a big concern. By Table 4 in |21j . 
under the image size of 128 x 128 with a 5 x 8 Gabor filter bank, the histogram 
extraction of LGBP takes around 0.45 second, S[LGBP_ Mag-|-LGXP] takes 0.99 
second. Extracting GOM feature takes 0.7 second [5]. But for our method, the 
‘feature extraction’ time is 0 since only raw phase is used for matching; the 
demodulation is the only required on-line computation of the probe face and 
it is extremely fast. Our Matlab implementation executes the matching of a 
face pair in 0.05 second in average (Gabor filtering included) on a 3.4GHz Intel 
GPU. We can therefore safely conclude that our GPBM outperforms the best 
Gabor-phase based approach (S[LGBP_ Mag-|-LGXP]) in efficiency for a big 
margin and we can also infer that the other methods in Table [3] could hardly 
be more efficient than our GPBM due to higher image resolution, Gabor face 
dimensions, and additional photometric processing. Here we should mention that 
our GPBM used the very basic ‘exhaust search’ strategy which is not an efficient 
choice for the matching. Our preliminary experiments show that there is a large 
potential to achieve even higher efficiency by using optimized searching strategy. 
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(Optimization of the Block Matching framework is outside the scope of this 
paper.) 

5 Conclusions 

In this paper, we propose a plain approach to leverage the demodulated Gabor 
phase for face identification based on the Block Matching method. The proposed 
approach dose not utilize a large Gabor filter bank or any training process. 
Rather, it only depends on the signal representation from a single-scale Gabor 
filter pair and performs explicit matching over the raw Gabor phase spectrum. 

Gomparative experiments show that: 1) our approach features the highest 
accuracy utilizing the Gabor phase for face recognition; 2) our approach has very 
low computational complexity with totally comparable performance to state- 
of-the-art methods. Our experiments also confirm that the Gabor phase is a 
powerful source to construct features for face identification. To leverage the 
power of the Gabor phase, the key is to have a suitable feature construction 
approach. In this paper, we show that the Blocking Matching is a good choice 
for such purpose. We strongly recommend the Block Matching as an alternative 
to the commonly adopted LBP should be used more in face recognition. 
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